An objective and subjective evaluation of content-based privacy protection of face images in video surveillance systems using JPEG XR

An objective and subjective evaluation of content-based privacy protection of face images in video surveillance systems using JPEG XR Hosik Sohn, Dohyoung Lee, Wesley De Neve, Konstantinos N. Plataniotis, and Yong Man Ro Image and Video Systems Lab, Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Yuseong-gu, Daejeon, 305-701, Korea Multimedia Lab, The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, M5S 3GA, Canada Abstract The increasing ability of video surveillance systems to successfully recognize faces has recently raised privacy concerns. To mitigate these concerns, several tools have been developed to conceal the identity of face images. However, little attention has thus far been paid to a profound evaluation of the effectiveness of privacy protection tools, thus leaving room for achieving a better understanding of experimental conditions that may cause privacy leakage and the effectiveness of different approaches for evaluating the level of security offered by privacy protection tools. To that end, in this chapter, we make use of both objective and subjective assessments to analyze the privacy-preserving nature of a layered scrambling technique developed for video surveillance systems using JPEG Extended Range (JPEG XR). In our objective assessments, we apply automatic face recognition (FR) techniques and general-purpose visual security metrics to scrambled face images, testing their effectiveness under varying experimental conditions. In our subjective assessments, we study whether agreement exists between the judgments of human observers and the output of automatic FR. Our results show that the scrambled face images come, in general, with a feasible level of protection against automatic and human FR. However, for video surveillance requiring a high level of privacy protection, our results indicate that the strength of the scrambling technique studied needs to be enhanced at low bit rates, that chroma information needs to be scrambled, and that the presence of eye glasses and a low number of gallery face images may contribute to the success of a replacement attack. Our results also demonstrate that, compared to automatic FR, the general-purpose visual security metrics studied are less suited for detecting weaknesses in tools that aim at identity concealment. Additionally, our results show that our objective and subjective assessments are not always in agreement.